Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconcile with dev-access-esm1.6 - util- snmin threshold thru namelist #552

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

JhanSrbinovsky
Copy link
Collaborator

@JhanSrbinovsky JhanSrbinovsky commented Feb 25, 2025

Description

In ESM1.6 we have upgraded snmin(the threshold snow depth triggering 3-layer treatment) to a runtime configurable constant.

Merging ESM1.6 code we already have modifications to cable_common and cable_runtime_opts. Strictly speaking, it shouldn't be necessary to make any mods to offline at all.

However, with some very minor modifications (on top of those that are already coming through the merge anyway), we can implement snmin as a runtime configurable constant in offline as well.

put snmin=1.0 in cable.nml to reproduce main.

Type of change

Enhancement following ESM1.6

Checklist

  • The new content is accessible and located in the appropriate section
  • I have checked that links are valid and point to the intended content
  • I have checked my code/text and corrected any misspellings

Testing

  • Are the changes bitwise-compatible with the main branch? If working on an optional feature, are the results bitwise-compatible when this feature is off? If yes, copy benchcab output showing successful completion of the bitwise compatibility tests or equivalent results below this line.

2025-03-03 16:40:25,871 - INFO - benchcab.benchcab.py:391 - 0 failed, 20 passed


📚 Documentation preview 📚: https://cable--552.org.readthedocs.build/en/552/

@JhanSrbinovsky JhanSrbinovsky changed the title snmin thru nml. iput snmin=0.11 in cable.nml to reproduce main. snmin thru namelist Feb 26, 2025
@JhanSrbinovsky
Copy link
Collaborator Author

@ccarouge benchcab is failing. can you see /g/data/p66/jxs599/bench_example/benchmark_cable_qsub.sh.o135997354? same error twice with change 42 site to 5 site test

@JhanSrbinovsky JhanSrbinovsky changed the title snmin thru namelist reconcile with dev-access-esm1.6 - util- snmin threshold thru namelist Feb 26, 2025
@SeanBryan51
Copy link
Collaborator

Hi @JhanSrbinovsky, I don't have access to /g/data/p66/jxs599/bench_example/benchmark_cable_qsub.sh.o135997354 due to file permissions. What does your benchcab configuration file look like? I may be able to reproduce your errors that way and take a look.

@JhanSrbinovsky
Copy link
Collaborator Author

@SeanBryan51 I've cp-ed the whole directory to here:

/scratch/public/jxs599/bench_example

@SeanBryan51
Copy link
Collaborator

SeanBryan51 commented Feb 27, 2025

/scratch/public/jxs599/bench_example

@JhanSrbinovsky still no luck, can you check the permissions on the directory?

$ ls -la /scratch/public/jxs599/bench_example
ls: cannot access '/scratch/public/jxs599/bench_example': Permission denied

@JhanSrbinovsky
Copy link
Collaborator Author

Should be OK – I though //scratch/public forced a umask on whatever you put there.

@SeanBryan51
Copy link
Collaborator

I've so far had no issues testing sync_ESM16_util with the five site test. I am currently running the 42 sites and will report back.

@JhanSrbinovsky
Copy link
Collaborator Author

@SeanBryan51 sorry I wasn't clear. It appears to run - but checking PBS log files for reporting on bitwise compatability - It isnt there. checking runs/fluxsite/outputs/ - also empty

this is entire file

"benchmark_cable_qsub.sh.o136083597" 35L, 2459C 35,1 All
ERROR: Directory '/g/data/hr22/modulefiles' not found
ERROR: Unable to locate a modulefile for 'cylc7/23.09-cdev'

/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/bin/benchcab fluxsite-run-tasks --config=config.yaml
Traceback (most recent call last):
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/bin/benchcab", line 10, in
sys.exit(main())
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/benchcab/main.py", line 45, in main
parse_and_dispatch(parser, app)
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/benchcab/main.py", line 35, in parse_and_dispatch
func(**args)
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/benchcab/benchcab.py", line 348, in fluxsite_run_tasks
config = self._get_config(config_path)
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/benchcab/benchcab.py", line 152, in _get_config
self._config = read_config(config_path)
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/benchcab/config.py", line 170, in read_config
config = read_config_file(config_path)
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/benchcab/config.py", line 144, in read_config_file
with Path.open(Path(config_path), "r", encoding="utf-8") as file:
File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: 'config.yaml'

======================================================================================
Resource Usage on 2025-02-28 12:29:29:
Job Id: 136083597.gadi-pbs
Project: p66
Exit Status: 1
Service Units: 0.14
NCPUs Requested: 18 NCPUs Used: 18
CPU Time Used: 00:00:03
Memory Requested: 30.0GB Memory Used: 359.02MB
Walltime requested: 06:00:00 Walltime Used: 00:00:14
JobFS requested: 100.0MB JobFS used: 0B

@JhanSrbinovsky
Copy link
Collaborator Author

update -the payu laboratory seem to have worked

@SeanBryan51
Copy link
Collaborator

I seem to get different output, and yes I am also getting differences between model outputs. From running the 42 sites I get the following summary in the PBS log file:

2025-02-28 12:58:16,731 - INFO - benchcab.benchcab.py:380 - Running comparison tasks...
2025-02-28 12:58:16,754 - INFO - benchcab.benchcab.py:381 - tasks: 168 (models: 2, sites: 42, science configurations: 4)
2025-02-28 12:58:25,380 - INFO - comparison.comparison.py:73 - Success: files AU-GWW_2013-2017_OzFlux_Met_R0_S1_out.nc AU-GWW_2013-2017_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:25,555 - INFO - comparison.comparison.py:73 - Success: files AU-GWW_2013-2017_OzFlux_Met_R0_S0_out.nc AU-GWW_2013-2017_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:27,116 - INFO - comparison.comparison.py:73 - Success: files AU-Cum_2013-2018_OzFlux_Met_R0_S2_out.nc AU-Cum_2013-2018_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:27,308 - INFO - comparison.comparison.py:73 - Success: files AU-Cum_2013-2018_OzFlux_Met_R0_S0_out.nc AU-Cum_2013-2018_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:27,401 - INFO - comparison.comparison.py:73 - Success: files AU-Cum_2013-2018_OzFlux_Met_R0_S1_out.nc AU-Cum_2013-2018_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:27,434 - INFO - comparison.comparison.py:73 - Success: files AU-Cum_2013-2018_OzFlux_Met_R0_S3_out.nc AU-Cum_2013-2018_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:28,821 - INFO - comparison.comparison.py:73 - Success: files AU-ASM_2011-2017_OzFlux_Met_R0_S1_out.nc AU-ASM_2011-2017_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:28,936 - INFO - comparison.comparison.py:73 - Success: files AU-ASM_2011-2017_OzFlux_Met_R0_S2_out.nc AU-ASM_2011-2017_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:29,002 - INFO - comparison.comparison.py:73 - Success: files AU-ASM_2011-2017_OzFlux_Met_R0_S0_out.nc AU-ASM_2011-2017_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:29,106 - INFO - comparison.comparison.py:73 - Success: files AU-ASM_2011-2017_OzFlux_Met_R0_S3_out.nc AU-ASM_2011-2017_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:30,528 - INFO - comparison.comparison.py:73 - Success: files AU-Tum_2002-2017_OzFlux_Met_R0_S2_out.nc AU-Tum_2002-2017_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:30,611 - INFO - comparison.comparison.py:73 - Success: files AU-Tum_2002-2017_OzFlux_Met_R0_S1_out.nc AU-Tum_2002-2017_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:30,614 - INFO - comparison.comparison.py:73 - Success: files AU-Tum_2002-2017_OzFlux_Met_R0_S0_out.nc AU-Tum_2002-2017_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:30,809 - INFO - comparison.comparison.py:73 - Success: files AU-Tum_2002-2017_OzFlux_Met_R0_S3_out.nc AU-Tum_2002-2017_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:34,119 - INFO - comparison.comparison.py:73 - Success: files AU-GWW_2013-2017_OzFlux_Met_R0_S2_out.nc AU-GWW_2013-2017_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:34,224 - INFO - comparison.comparison.py:73 - Success: files AU-GWW_2013-2017_OzFlux_Met_R0_S3_out.nc AU-GWW_2013-2017_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:35,921 - INFO - comparison.comparison.py:73 - Success: files BR-Sa3_2001-2003_FLUXNET2015_Met_R0_S2_out.nc BR-Sa3_2001-2003_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:58:36,000 - INFO - comparison.comparison.py:73 - Success: files BR-Sa3_2001-2003_FLUXNET2015_Met_R0_S0_out.nc BR-Sa3_2001-2003_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:58:36,140 - INFO - comparison.comparison.py:73 - Success: files BR-Sa3_2001-2003_FLUXNET2015_Met_R0_S3_out.nc BR-Sa3_2001-2003_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:58:36,157 - INFO - comparison.comparison.py:73 - Success: files BR-Sa3_2001-2003_FLUXNET2015_Met_R0_S1_out.nc BR-Sa3_2001-2003_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:58:41,141 - INFO - comparison.comparison.py:73 - Success: files AU-Ctr_2010-2017_OzFlux_Met_R0_S0_out.nc AU-Ctr_2010-2017_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:41,367 - INFO - comparison.comparison.py:73 - Success: files AU-Ctr_2010-2017_OzFlux_Met_R0_S1_out.nc AU-Ctr_2010-2017_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:41,848 - INFO - comparison.comparison.py:73 - Success: files AU-Ctr_2010-2017_OzFlux_Met_R0_S3_out.nc AU-Ctr_2010-2017_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:41,967 - INFO - comparison.comparison.py:73 - Success: files AU-Ctr_2010-2017_OzFlux_Met_R0_S2_out.nc AU-Ctr_2010-2017_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:42,883 - INFO - comparison.comparison.py:73 - Success: files AU-Stp_2010-2017_OzFlux_Met_R0_S0_out.nc AU-Stp_2010-2017_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:43,081 - INFO - comparison.comparison.py:73 - Success: files AU-How_2003-2017_OzFlux_Met_R0_S0_out.nc AU-How_2003-2017_OzFlux_Met_R1_S0_out.nc are identical
2025-02-28 12:58:43,122 - INFO - comparison.comparison.py:73 - Success: files AU-How_2003-2017_OzFlux_Met_R0_S3_out.nc AU-How_2003-2017_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:43,267 - INFO - comparison.comparison.py:73 - Success: files AU-How_2003-2017_OzFlux_Met_R0_S1_out.nc AU-How_2003-2017_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:43,422 - INFO - comparison.comparison.py:73 - Success: files AU-Stp_2010-2017_OzFlux_Met_R0_S3_out.nc AU-Stp_2010-2017_OzFlux_Met_R1_S3_out.nc are identical
2025-02-28 12:58:43,531 - INFO - comparison.comparison.py:73 - Success: files AU-Stp_2010-2017_OzFlux_Met_R0_S1_out.nc AU-Stp_2010-2017_OzFlux_Met_R1_S1_out.nc are identical
2025-02-28 12:58:43,541 - INFO - comparison.comparison.py:73 - Success: files AU-Stp_2010-2017_OzFlux_Met_R0_S2_out.nc AU-Stp_2010-2017_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:43,874 - INFO - comparison.comparison.py:73 - Success: files AU-How_2003-2017_OzFlux_Met_R0_S2_out.nc AU-How_2003-2017_OzFlux_Met_R1_S2_out.nc are identical
2025-02-28 12:58:46,861 - INFO - comparison.comparison.py:73 - Success: files CA-Qfo_2004-2010_FLUXNET2015_Met_R0_S0_out.nc CA-Qfo_2004-2010_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:58:46,890 - INFO - comparison.comparison.py:73 - Success: files CA-Qfo_2004-2010_FLUXNET2015_Met_R0_S1_out.nc CA-Qfo_2004-2010_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:58:47,487 - INFO - comparison.comparison.py:73 - Success: files CN-Cha_2003-2005_FLUXNET2015_Met_R0_S0_out.nc CN-Cha_2003-2005_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:58:47,709 - INFO - comparison.comparison.py:73 - Success: files CN-Cha_2003-2005_FLUXNET2015_Met_R0_S1_out.nc CN-Cha_2003-2005_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:58:48,198 - INFO - comparison.comparison.py:73 - Success: files CN-Cha_2003-2005_FLUXNET2015_Met_R0_S2_out.nc CN-Cha_2003-2005_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:58:48,601 - INFO - comparison.comparison.py:73 - Success: files CA-Qfo_2004-2010_FLUXNET2015_Met_R0_S3_out.nc CA-Qfo_2004-2010_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:58:48,656 - INFO - comparison.comparison.py:73 - Success: files CN-Din_2003-2005_FLUXNET2015_Met_R0_S0_out.nc CN-Din_2003-2005_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:58:49,018 - INFO - comparison.comparison.py:73 - Success: files CN-Cha_2003-2005_FLUXNET2015_Met_R0_S3_out.nc CN-Cha_2003-2005_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:58:49,070 - INFO - comparison.comparison.py:73 - Success: files CN-Din_2003-2005_FLUXNET2015_Met_R0_S2_out.nc CN-Din_2003-2005_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:58:49,073 - INFO - comparison.comparison.py:73 - Success: files CN-Din_2003-2005_FLUXNET2015_Met_R0_S3_out.nc CN-Din_2003-2005_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:58:49,219 - INFO - comparison.comparison.py:73 - Success: files CN-Din_2003-2005_FLUXNET2015_Met_R0_S1_out.nc CN-Din_2003-2005_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:58:51,165 - INFO - comparison.comparison.py:73 - Success: files CA-Qfo_2004-2010_FLUXNET2015_Met_R0_S2_out.nc CA-Qfo_2004-2010_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:07,348 - INFO - comparison.comparison.py:73 - Success: files DE-Gri_2004-2014_FLUXNET2015_Met_R0_S0_out.nc DE-Gri_2004-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:07,609 - INFO - comparison.comparison.py:73 - Success: files DE-Gri_2004-2014_FLUXNET2015_Met_R0_S2_out.nc DE-Gri_2004-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:07,973 - INFO - comparison.comparison.py:73 - Success: files DE-Gri_2004-2014_FLUXNET2015_Met_R0_S1_out.nc DE-Gri_2004-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:08,036 - INFO - comparison.comparison.py:73 - Success: files CH-Dav_1997-2014_FLUXNET2015_Met_R0_S0_out.nc CH-Dav_1997-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:08,726 - INFO - comparison.comparison.py:73 - Success: files DE-Geb_2001-2014_FLUXNET2015_Met_R0_S0_out.nc DE-Geb_2001-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:08,760 - INFO - comparison.comparison.py:73 - Success: files DE-Gri_2004-2014_FLUXNET2015_Met_R0_S3_out.nc DE-Gri_2004-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:09,047 - INFO - comparison.comparison.py:73 - Success: files DE-Geb_2001-2014_FLUXNET2015_Met_R0_S1_out.nc DE-Geb_2001-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:10,654 - INFO - comparison.comparison.py:73 - Success: files CH-Dav_1997-2014_FLUXNET2015_Met_R0_S1_out.nc CH-Dav_1997-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:11,807 - INFO - comparison.comparison.py:73 - Success: files DE-Geb_2001-2014_FLUXNET2015_Met_R0_S3_out.nc DE-Geb_2001-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:11,874 - INFO - comparison.comparison.py:73 - Success: files DE-Geb_2001-2014_FLUXNET2015_Met_R0_S2_out.nc DE-Geb_2001-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:12,060 - INFO - comparison.comparison.py:73 - Success: files DE-Hai_2000-2012_FLUXNET2015_Met_R0_S0_out.nc DE-Hai_2000-2012_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:12,468 - INFO - comparison.comparison.py:73 - Success: files DE-Hai_2000-2012_FLUXNET2015_Met_R0_S1_out.nc DE-Hai_2000-2012_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:12,763 - INFO - comparison.comparison.py:73 - Success: files DE-Hai_2000-2012_FLUXNET2015_Met_R0_S3_out.nc DE-Hai_2000-2012_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:12,910 - INFO - comparison.comparison.py:73 - Success: files DE-Hai_2000-2012_FLUXNET2015_Met_R0_S2_out.nc DE-Hai_2000-2012_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:13,172 - INFO - comparison.comparison.py:73 - Success: files CH-Dav_1997-2014_FLUXNET2015_Met_R0_S3_out.nc CH-Dav_1997-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:13,883 - INFO - comparison.comparison.py:73 - Success: files CH-Dav_1997-2014_FLUXNET2015_Met_R0_S2_out.nc CH-Dav_1997-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:19,461 - INFO - comparison.comparison.py:73 - Success: files DE-Tha_1998-2014_FLUXNET2015_Met_R0_S0_out.nc DE-Tha_1998-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:21,749 - INFO - comparison.comparison.py:73 - Success: files DE-Tha_1998-2014_FLUXNET2015_Met_R0_S1_out.nc DE-Tha_1998-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:28,152 - INFO - comparison.comparison.py:73 - Success: files FR-Gri_2005-2013_FLUXNET2015_Met_R0_S0_out.nc FR-Gri_2005-2013_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:28,525 - INFO - comparison.comparison.py:73 - Success: files FR-Gri_2005-2013_FLUXNET2015_Met_R0_S1_out.nc FR-Gri_2005-2013_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:29,024 - INFO - comparison.comparison.py:73 - Success: files FR-Gri_2005-2013_FLUXNET2015_Met_R0_S2_out.nc FR-Gri_2005-2013_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:30,080 - INFO - comparison.comparison.py:73 - Success: files FR-Gri_2005-2013_FLUXNET2015_Met_R0_S3_out.nc FR-Gri_2005-2013_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:37,416 - INFO - comparison.comparison.py:73 - Success: files DE-Tha_1998-2014_FLUXNET2015_Met_R0_S2_out.nc DE-Tha_1998-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:37,720 - INFO - comparison.comparison.py:73 - Success: files DE-Tha_1998-2014_FLUXNET2015_Met_R0_S3_out.nc DE-Tha_1998-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:40,077 - INFO - comparison.comparison.py:73 - Success: files DK-Sor_1997-2014_FLUXNET2015_Met_R0_S1_out.nc DK-Sor_1997-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:40,168 - INFO - comparison.comparison.py:73 - Success: files DK-Sor_1997-2014_FLUXNET2015_Met_R0_S0_out.nc DK-Sor_1997-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:40,173 - INFO - comparison.comparison.py:73 - Success: files FR-Pue_2000-2014_FLUXNET2015_Met_R0_S0_out.nc FR-Pue_2000-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:41,001 - INFO - comparison.comparison.py:73 - Success: files DK-Sor_1997-2014_FLUXNET2015_Met_R0_S3_out.nc DK-Sor_1997-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:41,343 - INFO - comparison.comparison.py:73 - Success: files FR-Pue_2000-2014_FLUXNET2015_Met_R0_S1_out.nc FR-Pue_2000-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:41,446 - INFO - comparison.comparison.py:73 - Success: files DK-Sor_1997-2014_FLUXNET2015_Met_R0_S2_out.nc DK-Sor_1997-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:43,215 - INFO - comparison.comparison.py:73 - Success: files FI-Hyy_1996-2014_FLUXNET2015_Met_R0_S0_out.nc FI-Hyy_1996-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:45,205 - INFO - comparison.comparison.py:73 - Success: files FI-Hyy_1996-2014_FLUXNET2015_Met_R0_S1_out.nc FI-Hyy_1996-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:45,831 - INFO - comparison.comparison.py:73 - Success: files FR-Pue_2000-2014_FLUXNET2015_Met_R0_S2_out.nc FR-Pue_2000-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:45,917 - INFO - comparison.comparison.py:73 - Success: files FI-Hyy_1996-2014_FLUXNET2015_Met_R0_S3_out.nc FI-Hyy_1996-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:46,458 - INFO - comparison.comparison.py:73 - Success: files FI-Hyy_1996-2014_FLUXNET2015_Met_R0_S2_out.nc FI-Hyy_1996-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:48,036 - INFO - comparison.comparison.py:73 - Success: files GF-Guy_2004-2014_FLUXNET2015_Met_R0_S0_out.nc GF-Guy_2004-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:48,161 - INFO - comparison.comparison.py:73 - Success: files GF-Guy_2004-2014_FLUXNET2015_Met_R0_S1_out.nc GF-Guy_2004-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:48,928 - INFO - comparison.comparison.py:73 - Success: files GF-Guy_2004-2014_FLUXNET2015_Met_R0_S2_out.nc GF-Guy_2004-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:49,408 - INFO - comparison.comparison.py:73 - Success: files FR-Pue_2000-2014_FLUXNET2015_Met_R0_S3_out.nc FR-Pue_2000-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:49,945 - INFO - comparison.comparison.py:73 - Success: files GF-Guy_2004-2014_FLUXNET2015_Met_R0_S3_out.nc GF-Guy_2004-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 12:59:57,967 - INFO - comparison.comparison.py:73 - Success: files IT-MBo_2003-2012_FLUXNET2015_Met_R0_S0_out.nc IT-MBo_2003-2012_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 12:59:59,003 - INFO - comparison.comparison.py:73 - Success: files IT-MBo_2003-2012_FLUXNET2015_Met_R0_S1_out.nc IT-MBo_2003-2012_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 12:59:59,095 - ERROR - comparison.comparison.py:81 - Failure: files IT-Lav_2005-2014_FLUXNET2015_Met_R0_S1_out.nc IT-Lav_2005-2014_FLUXNET2015_Met_R1_S1_out.nc differ. 
2025-02-28 12:59:59,096 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/IT-Lav_2005-2014_FLUXNET2015_Met_S1_R0_R1.txt
2025-02-28 12:59:59,341 - INFO - comparison.comparison.py:73 - Success: files IT-MBo_2003-2012_FLUXNET2015_Met_R0_S2_out.nc IT-MBo_2003-2012_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 12:59:59,834 - INFO - comparison.comparison.py:73 - Success: files IT-MBo_2003-2012_FLUXNET2015_Met_R0_S3_out.nc IT-MBo_2003-2012_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:02,775 - ERROR - comparison.comparison.py:81 - Failure: files IT-Lav_2005-2014_FLUXNET2015_Met_R0_S0_out.nc IT-Lav_2005-2014_FLUXNET2015_Met_R1_S0_out.nc differ. 
2025-02-28 13:00:02,776 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/IT-Lav_2005-2014_FLUXNET2015_Met_S0_R0_R1.txt
2025-02-28 13:00:03,650 - ERROR - comparison.comparison.py:81 - Failure: files IT-Lav_2005-2014_FLUXNET2015_Met_R0_S2_out.nc IT-Lav_2005-2014_FLUXNET2015_Met_R1_S2_out.nc differ. 
2025-02-28 13:00:03,650 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/IT-Lav_2005-2014_FLUXNET2015_Met_S2_R0_R1.txt
2025-02-28 13:00:03,766 - INFO - comparison.comparison.py:73 - Success: files IT-Noe_2004-2014_FLUXNET2015_Met_R0_S0_out.nc IT-Noe_2004-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:00:05,338 - INFO - comparison.comparison.py:73 - Success: files IT-Noe_2004-2014_FLUXNET2015_Met_R0_S1_out.nc IT-Noe_2004-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:00:05,771 - ERROR - comparison.comparison.py:81 - Failure: files IT-Lav_2005-2014_FLUXNET2015_Met_R0_S3_out.nc IT-Lav_2005-2014_FLUXNET2015_Met_R1_S3_out.nc differ. 
2025-02-28 13:00:05,771 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/IT-Lav_2005-2014_FLUXNET2015_Met_S3_R0_R1.txt
2025-02-28 13:00:05,846 - INFO - comparison.comparison.py:73 - Success: files IT-Noe_2004-2014_FLUXNET2015_Met_R0_S2_out.nc IT-Noe_2004-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:00:06,540 - INFO - comparison.comparison.py:73 - Success: files IT-Noe_2004-2014_FLUXNET2015_Met_R0_S3_out.nc IT-Noe_2004-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:10,950 - INFO - comparison.comparison.py:73 - Success: files RU-Fyo_2003-2014_FLUXNET2015_Met_R0_S0_out.nc RU-Fyo_2003-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:00:11,813 - INFO - comparison.comparison.py:73 - Success: files RU-Fyo_2003-2014_FLUXNET2015_Met_R0_S1_out.nc RU-Fyo_2003-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:00:12,127 - INFO - comparison.comparison.py:73 - Success: files US-Blo_2000-2006_FLUXNET2015_Met_R0_S0_out.nc US-Blo_2000-2006_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:00:12,136 - INFO - comparison.comparison.py:73 - Success: files US-Blo_2000-2006_FLUXNET2015_Met_R0_S1_out.nc US-Blo_2000-2006_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:00:12,878 - INFO - comparison.comparison.py:73 - Success: files US-Blo_2000-2006_FLUXNET2015_Met_R0_S2_out.nc US-Blo_2000-2006_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:00:15,572 - ERROR - comparison.comparison.py:81 - Failure: files US-GLE_2009-2014_FLUXNET2015_Met_R0_S1_out.nc US-GLE_2009-2014_FLUXNET2015_Met_R1_S1_out.nc differ. 
2025-02-28 13:00:15,572 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-GLE_2009-2014_FLUXNET2015_Met_S1_R0_R1.txt
2025-02-28 13:00:15,780 - INFO - comparison.comparison.py:73 - Success: files US-Blo_2000-2006_FLUXNET2015_Met_R0_S3_out.nc US-Blo_2000-2006_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:16,529 - INFO - comparison.comparison.py:73 - Success: files NL-Loo_1997-2013_FLUXNET2015_Met_R0_S0_out.nc NL-Loo_1997-2013_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:00:17,720 - ERROR - comparison.comparison.py:81 - Failure: files US-GLE_2009-2014_FLUXNET2015_Met_R0_S3_out.nc US-GLE_2009-2014_FLUXNET2015_Met_R1_S3_out.nc differ. 
2025-02-28 13:00:17,720 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-GLE_2009-2014_FLUXNET2015_Met_S3_R0_R1.txt
2025-02-28 13:00:18,396 - INFO - comparison.comparison.py:73 - Success: files NL-Loo_1997-2013_FLUXNET2015_Met_R0_S1_out.nc NL-Loo_1997-2013_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:00:19,035 - INFO - comparison.comparison.py:73 - Success: files NL-Loo_1997-2013_FLUXNET2015_Met_R0_S2_out.nc NL-Loo_1997-2013_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:00:20,124 - INFO - comparison.comparison.py:73 - Success: files NL-Loo_1997-2013_FLUXNET2015_Met_R0_S3_out.nc NL-Loo_1997-2013_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:20,317 - INFO - comparison.comparison.py:73 - Success: files RU-Fyo_2003-2014_FLUXNET2015_Met_R0_S3_out.nc RU-Fyo_2003-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:22,113 - INFO - comparison.comparison.py:73 - Success: files RU-Fyo_2003-2014_FLUXNET2015_Met_R0_S2_out.nc RU-Fyo_2003-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:00:26,009 - ERROR - comparison.comparison.py:81 - Failure: files US-GLE_2009-2014_FLUXNET2015_Met_R0_S0_out.nc US-GLE_2009-2014_FLUXNET2015_Met_R1_S0_out.nc differ. 
2025-02-28 13:00:26,009 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-GLE_2009-2014_FLUXNET2015_Met_S0_R0_R1.txt
2025-02-28 13:00:26,923 - INFO - comparison.comparison.py:73 - Success: files US-Myb_2011-2014_FLUXNET2015_Met_R0_S0_out.nc US-Myb_2011-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:00:27,673 - ERROR - comparison.comparison.py:81 - Failure: files US-Ha1_1992-2012_FLUXNET2015_Met_R0_S0_out.nc US-Ha1_1992-2012_FLUXNET2015_Met_R1_S0_out.nc differ. 
2025-02-28 13:00:27,673 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Ha1_1992-2012_FLUXNET2015_Met_S0_R0_R1.txt
2025-02-28 13:00:28,164 - INFO - comparison.comparison.py:73 - Success: files US-Myb_2011-2014_FLUXNET2015_Met_R0_S1_out.nc US-Myb_2011-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:00:30,123 - ERROR - comparison.comparison.py:81 - Failure: files US-Ha1_1992-2012_FLUXNET2015_Met_R0_S1_out.nc US-Ha1_1992-2012_FLUXNET2015_Met_R1_S1_out.nc differ. 
2025-02-28 13:00:30,123 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Ha1_1992-2012_FLUXNET2015_Met_S1_R0_R1.txt
2025-02-28 13:00:30,245 - INFO - comparison.comparison.py:73 - Success: files US-Myb_2011-2014_FLUXNET2015_Met_R0_S2_out.nc US-Myb_2011-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:00:31,865 - INFO - comparison.comparison.py:73 - Success: files US-Myb_2011-2014_FLUXNET2015_Met_R0_S3_out.nc US-Myb_2011-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:32,220 - ERROR - comparison.comparison.py:81 - Failure: files US-GLE_2009-2014_FLUXNET2015_Met_R0_S2_out.nc US-GLE_2009-2014_FLUXNET2015_Met_R1_S2_out.nc differ. 
2025-02-28 13:00:32,220 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-GLE_2009-2014_FLUXNET2015_Met_S2_R0_R1.txt
2025-02-28 13:00:33,482 - ERROR - comparison.comparison.py:81 - Failure: files US-Ha1_1992-2012_FLUXNET2015_Met_R0_S2_out.nc US-Ha1_1992-2012_FLUXNET2015_Met_R1_S2_out.nc differ. 
2025-02-28 13:00:33,482 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Ha1_1992-2012_FLUXNET2015_Met_S2_R0_R1.txt
2025-02-28 13:00:36,687 - ERROR - comparison.comparison.py:81 - Failure: files US-Ha1_1992-2012_FLUXNET2015_Met_R0_S3_out.nc US-Ha1_1992-2012_FLUXNET2015_Met_R1_S3_out.nc differ. 
2025-02-28 13:00:36,688 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Ha1_1992-2012_FLUXNET2015_Met_S3_R0_R1.txt
2025-02-28 13:00:37,431 - ERROR - comparison.comparison.py:81 - Failure: files US-MMS_1999-2014_FLUXNET2015_Met_R0_S0_out.nc US-MMS_1999-2014_FLUXNET2015_Met_R1_S0_out.nc differ. 
2025-02-28 13:00:37,432 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-MMS_1999-2014_FLUXNET2015_Met_S0_R0_R1.txt
2025-02-28 13:00:37,986 - ERROR - comparison.comparison.py:81 - Failure: files US-MMS_1999-2014_FLUXNET2015_Met_R0_S1_out.nc US-MMS_1999-2014_FLUXNET2015_Met_R1_S1_out.nc differ. 
2025-02-28 13:00:37,986 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-MMS_1999-2014_FLUXNET2015_Met_S1_R0_R1.txt
2025-02-28 13:00:44,014 - ERROR - comparison.comparison.py:81 - Failure: files US-MMS_1999-2014_FLUXNET2015_Met_R0_S2_out.nc US-MMS_1999-2014_FLUXNET2015_Met_R1_S2_out.nc differ. 
2025-02-28 13:00:44,015 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-MMS_1999-2014_FLUXNET2015_Met_S2_R0_R1.txt
2025-02-28 13:00:46,050 - ERROR - comparison.comparison.py:81 - Failure: files US-MMS_1999-2014_FLUXNET2015_Met_R0_S3_out.nc US-MMS_1999-2014_FLUXNET2015_Met_R1_S3_out.nc differ. 
2025-02-28 13:00:46,051 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-MMS_1999-2014_FLUXNET2015_Met_S3_R0_R1.txt
2025-02-28 13:00:46,366 - ERROR - comparison.comparison.py:81 - Failure: files US-Me2_2002-2014_FLUXNET2015_Met_R0_S0_out.nc US-Me2_2002-2014_FLUXNET2015_Met_R1_S0_out.nc differ. 
2025-02-28 13:00:46,366 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Me2_2002-2014_FLUXNET2015_Met_S0_R0_R1.txt
2025-02-28 13:00:49,581 - INFO - comparison.comparison.py:73 - Success: files US-PFa_1995-2014_FLUXNET2015_Met_R0_S2_out.nc US-PFa_1995-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:00:51,202 - INFO - comparison.comparison.py:73 - Success: files US-PFa_1995-2014_FLUXNET2015_Met_R0_S3_out.nc US-PFa_1995-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:00:52,428 - INFO - comparison.comparison.py:73 - Success: files US-PFa_1995-2014_FLUXNET2015_Met_R0_S0_out.nc US-PFa_1995-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:00:53,422 - ERROR - comparison.comparison.py:81 - Failure: files US-Me2_2002-2014_FLUXNET2015_Met_R0_S1_out.nc US-Me2_2002-2014_FLUXNET2015_Met_R1_S1_out.nc differ. 
2025-02-28 13:00:53,422 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Me2_2002-2014_FLUXNET2015_Met_S1_R0_R1.txt
2025-02-28 13:00:55,394 - ERROR - comparison.comparison.py:81 - Failure: files US-Me2_2002-2014_FLUXNET2015_Met_R0_S2_out.nc US-Me2_2002-2014_FLUXNET2015_Met_R1_S2_out.nc differ. 
2025-02-28 13:00:55,394 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Me2_2002-2014_FLUXNET2015_Met_S2_R0_R1.txt
2025-02-28 13:00:55,621 - INFO - comparison.comparison.py:73 - Success: files US-PFa_1995-2014_FLUXNET2015_Met_R0_S1_out.nc US-PFa_1995-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:00:56,480 - ERROR - comparison.comparison.py:81 - Failure: files US-Me2_2002-2014_FLUXNET2015_Met_R0_S3_out.nc US-Me2_2002-2014_FLUXNET2015_Met_R1_S3_out.nc differ. 
2025-02-28 13:00:56,481 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-Me2_2002-2014_FLUXNET2015_Met_S3_R0_R1.txt
2025-02-28 13:00:57,656 - ERROR - comparison.comparison.py:81 - Failure: files US-FPe_2000-2006_LaThuile_Met_R0_S0_out.nc US-FPe_2000-2006_LaThuile_Met_R1_S0_out.nc differ. 
2025-02-28 13:00:57,657 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-FPe_2000-2006_LaThuile_Met_S0_R0_R1.txt
2025-02-28 13:01:00,204 - ERROR - comparison.comparison.py:81 - Failure: files US-FPe_2000-2006_LaThuile_Met_R0_S2_out.nc US-FPe_2000-2006_LaThuile_Met_R1_S2_out.nc differ. 
2025-02-28 13:01:00,204 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-FPe_2000-2006_LaThuile_Met_S2_R0_R1.txt
2025-02-28 13:01:00,867 - ERROR - comparison.comparison.py:81 - Failure: files US-FPe_2000-2006_LaThuile_Met_R0_S3_out.nc US-FPe_2000-2006_LaThuile_Met_R1_S3_out.nc differ. 
2025-02-28 13:01:00,867 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-FPe_2000-2006_LaThuile_Met_S3_R0_R1.txt
2025-02-28 13:01:01,483 - ERROR - comparison.comparison.py:81 - Failure: files US-FPe_2000-2006_LaThuile_Met_R0_S1_out.nc US-FPe_2000-2006_LaThuile_Met_R1_S1_out.nc differ. 
2025-02-28 13:01:01,483 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-FPe_2000-2006_LaThuile_Met_S1_R0_R1.txt
2025-02-28 13:01:03,725 - INFO - comparison.comparison.py:73 - Success: files US-SRG_2009-2014_FLUXNET2015_Met_R0_S1_out.nc US-SRG_2009-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:04,509 - INFO - comparison.comparison.py:73 - Success: files US-SRG_2009-2014_FLUXNET2015_Met_R0_S2_out.nc US-SRG_2009-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:07,133 - INFO - comparison.comparison.py:73 - Success: files US-SRG_2009-2014_FLUXNET2015_Met_R0_S0_out.nc US-SRG_2009-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:09,002 - INFO - comparison.comparison.py:73 - Success: files US-SRM_2004-2014_FLUXNET2015_Met_R0_S1_out.nc US-SRM_2004-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:09,872 - INFO - comparison.comparison.py:73 - Success: files US-SRG_2009-2014_FLUXNET2015_Met_R0_S3_out.nc US-SRG_2009-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:10,432 - INFO - comparison.comparison.py:73 - Success: files US-SRM_2004-2014_FLUXNET2015_Met_R0_S3_out.nc US-SRM_2004-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:11,129 - INFO - comparison.comparison.py:73 - Success: files US-SRM_2004-2014_FLUXNET2015_Met_R0_S2_out.nc US-SRM_2004-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:11,694 - INFO - comparison.comparison.py:73 - Success: files US-SRM_2004-2014_FLUXNET2015_Met_R0_S0_out.nc US-SRM_2004-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:14,297 - INFO - comparison.comparison.py:73 - Success: files US-UMB_2000-2014_FLUXNET2015_Met_R0_S0_out.nc US-UMB_2000-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:14,753 - INFO - comparison.comparison.py:73 - Success: files US-UMB_2000-2014_FLUXNET2015_Met_R0_S1_out.nc US-UMB_2000-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:19,123 - INFO - comparison.comparison.py:73 - Success: files US-UMB_2000-2014_FLUXNET2015_Met_R0_S3_out.nc US-UMB_2000-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:21,387 - INFO - comparison.comparison.py:73 - Success: files US-Ton_2001-2014_FLUXNET2015_Met_R0_S0_out.nc US-Ton_2001-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:21,809 - INFO - comparison.comparison.py:73 - Success: files US-Ton_2001-2014_FLUXNET2015_Met_R0_S2_out.nc US-Ton_2001-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:22,356 - ERROR - comparison.comparison.py:81 - Failure: files US-NR1_1999-2014_FLUXNET2015_Met_R0_S3_out.nc US-NR1_1999-2014_FLUXNET2015_Met_R1_S3_out.nc differ. 
2025-02-28 13:01:22,357 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-NR1_1999-2014_FLUXNET2015_Met_S3_R0_R1.txt
2025-02-28 13:01:22,776 - INFO - comparison.comparison.py:73 - Success: files US-Whs_2008-2014_FLUXNET2015_Met_R0_S0_out.nc US-Whs_2008-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:23,098 - INFO - comparison.comparison.py:73 - Success: files US-UMB_2000-2014_FLUXNET2015_Met_R0_S2_out.nc US-UMB_2000-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:23,853 - ERROR - comparison.comparison.py:81 - Failure: files US-NR1_1999-2014_FLUXNET2015_Met_R0_S0_out.nc US-NR1_1999-2014_FLUXNET2015_Met_R1_S0_out.nc differ. 
2025-02-28 13:01:23,853 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-NR1_1999-2014_FLUXNET2015_Met_S0_R0_R1.txt
2025-02-28 13:01:24,082 - INFO - comparison.comparison.py:73 - Success: files US-Whs_2008-2014_FLUXNET2015_Met_R0_S1_out.nc US-Whs_2008-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:24,913 - INFO - comparison.comparison.py:73 - Success: files US-Ton_2001-2014_FLUXNET2015_Met_R0_S1_out.nc US-Ton_2001-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:24,976 - INFO - comparison.comparison.py:73 - Success: files US-Ton_2001-2014_FLUXNET2015_Met_R0_S3_out.nc US-Ton_2001-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:26,972 - INFO - comparison.comparison.py:73 - Success: files US-Whs_2008-2014_FLUXNET2015_Met_R0_S3_out.nc US-Whs_2008-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:27,969 - ERROR - comparison.comparison.py:81 - Failure: files US-NR1_1999-2014_FLUXNET2015_Met_R0_S2_out.nc US-NR1_1999-2014_FLUXNET2015_Met_R1_S2_out.nc differ. 
2025-02-28 13:01:27,970 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-NR1_1999-2014_FLUXNET2015_Met_S2_R0_R1.txt
2025-02-28 13:01:29,042 - ERROR - comparison.comparison.py:81 - Failure: files US-NR1_1999-2014_FLUXNET2015_Met_R0_S1_out.nc US-NR1_1999-2014_FLUXNET2015_Met_R1_S1_out.nc differ. 
2025-02-28 13:01:29,042 - ERROR - comparison.comparison.py:82 - Results of diff have been written to runs/fluxsite/analysis/bitwise-comparisons/US-NR1_1999-2014_FLUXNET2015_Met_S1_R0_R1.txt
2025-02-28 13:01:31,031 - INFO - comparison.comparison.py:73 - Success: files US-Whs_2008-2014_FLUXNET2015_Met_R0_S2_out.nc US-Whs_2008-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:32,153 - INFO - comparison.comparison.py:73 - Success: files US-Var_2001-2014_FLUXNET2015_Met_R0_S0_out.nc US-Var_2001-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:33,657 - INFO - comparison.comparison.py:73 - Success: files US-Var_2001-2014_FLUXNET2015_Met_R0_S1_out.nc US-Var_2001-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:34,329 - INFO - comparison.comparison.py:73 - Success: files US-Var_2001-2014_FLUXNET2015_Met_R0_S2_out.nc US-Var_2001-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:34,456 - INFO - comparison.comparison.py:73 - Success: files US-Var_2001-2014_FLUXNET2015_Met_R0_S3_out.nc US-Var_2001-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:38,038 - INFO - comparison.comparison.py:73 - Success: files US-Wkg_2005-2014_FLUXNET2015_Met_R0_S0_out.nc US-Wkg_2005-2014_FLUXNET2015_Met_R1_S0_out.nc are identical
2025-02-28 13:01:38,131 - INFO - comparison.comparison.py:73 - Success: files US-Wkg_2005-2014_FLUXNET2015_Met_R0_S1_out.nc US-Wkg_2005-2014_FLUXNET2015_Met_R1_S1_out.nc are identical
2025-02-28 13:01:38,227 - INFO - comparison.comparison.py:73 - Success: files US-Wkg_2005-2014_FLUXNET2015_Met_R0_S2_out.nc US-Wkg_2005-2014_FLUXNET2015_Met_R1_S2_out.nc are identical
2025-02-28 13:01:38,762 - INFO - comparison.comparison.py:73 - Success: files US-Wkg_2005-2014_FLUXNET2015_Met_R0_S3_out.nc US-Wkg_2005-2014_FLUXNET2015_Met_R1_S3_out.nc are identical
2025-02-28 13:01:38,833 - INFO - benchcab.benchcab.py:391 - 28 failed, 140 passed

@SeanBryan51
Copy link
Collaborator

It could be some state left from the previous run that could be messing up things. I recommend running benchcab clean all to reset bench_example directory and then try again with benchcab run

@JhanSrbinovsky
Copy link
Collaborator Author

I did the cleaning several times. I've just moved it to /scratch. I'm still waiting on jobs to finish before it write a .o file, but there are at least files in the output directory now.

Bugger, I was 100% expecting see al 42 sites "identical" so this is going to be the next problem

@JhanSrbinovsky
Copy link
Collaborator Author

just got the .o file and it looks like it should. It seems it doesnt like being on /g/data/ ?

@JhanSrbinovsky
Copy link
Collaborator Author

@SeanBryan51 - where does benchcab pickup call.nml from - because that is what the change is and if it doesnt pick up src/offline/cable.nml then it initialising a threshold with some random number - Im surprised there armed ,ore Failures

@ccarouge
Copy link
Member

ccarouge commented Mar 2, 2025

The 28 failures are due to the snow aging changes. I'm guessing that change is in main but not in this PR. If you compare to the root commit of this branch, it shouldn't give any changes.

The reason I know if I ran benchab on #551 and it showed the same puzzling differences. Traced it back to the extra changes in main rather than changes in the branch. But I haven't checked what's in this current PR yet.

@SeanBryan51
Copy link
Collaborator

@JhanSrbinovsky to answer your question benchcab picks up the namelist from bench_example, both main and your branch are tested with the same cable.nml which does not include the namelist parameter which you have introduced in this PR

@ccarouge
Copy link
Member

ccarouge commented Mar 2, 2025

@JhanSrbinovsky To test changes to the namelist, use the patch option in config.yaml. See the example here.

@JhanSrbinovsky
Copy link
Collaborator Author

JhanSrbinovsky commented Mar 3, 2025

@JhanSrbinovsky To test changes to the namelist, use the patch option in config.yaml. See the example here.

@ccarouge I suspect YYYY here means substituting snmin=1.0? It isnt a part of cable_user type though?

@JhanSrbinovsky
Copy link
Collaborator Author

The 28 failures are due to the snow aging changes. I'm guessing that change is in main but not in this PR. If you compare to the root commit of this branch, it shouldn't give any changes.

The reason I know if I ran benchab on #551 and it showed the same puzzling differences. Traced it back to the extra changes in main rather than changes in the branch. But I haven't checked what's in this current PR yet.

I think I already rebased the branch - so it picked that up. I'll try ro rerun it with the patch thing.

@JhanSrbinovsky
Copy link
Collaborator Author

hmmmm - looking in runs/......./tasks/....../cable.nml it doesnt look like

cable_user%exsting_feature is gonna do it.

@JhanSrbinovsky
Copy link
Collaborator Author

patch:
  cable:
    snmin: 1.0

seems to have added what I need to cable.nml, we'll wait for the output

@JhanSrbinovsky
Copy link
Collaborator Author

2025-03-03 16:40:25,871 - INFO - benchcab.benchcab.py:391 - 0 failed, 20 passed

OK - thats better. I mixed up the branches I was testing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants